When researchers make an inference on a population, they base their assumptions on collected data analyzed using exploratory data analysis. These assumptions can be beneficial in areas like medical research and have been used during the analysis of the COVID-19 crisis. However, making assumptions based on populations can sometimes lead to incorrect conclusions. There are two hypotheses when performing statistical testing. The first, the null hypothesis, states that there is no significant difference between populations, while the alternative hypothesis states that there are significant differences between populations. In this post, we examine Type I and Type II errors in hypothesis testing.
The first error which occurs in statistical inference is a Type I error. Type I errors occur when the null hypothesis is correct but is rejected. This leads to a false positive wherein the researcher concludes that there is a statistically significant difference when one does not exist. Type I errors are dangerous errors because wrong conclusions can lead to bad decisions.
The second error occurring in statistical inference is a Type II error. Type II errors occur when the null hypothesis is false but accepted. Type II errors are known as false negatives because they assume a statistically significant difference between populations where there is none. These errors can also lead to incorrect conclusions, which are costly to organizations.
For Type I errors, you should use a lower p-value, which is the probability of obtaining results as if the null hypothesis is correct. The alpha level (α) is the probability of rejecting the null hypothesis if it is true. Typically, α levels fall within either 95% or 99%. The corresponding p-values would be 0.05 and 0.01, respectively. The lower the p-value, the higher the chance of avoiding Type I errors. Mathematically, power is 1 – β, where β is the probability of making a Type II error. Power is the probability of rejecting the null hypothesis and accepting the alternative hypothesis. In a future THRIVE post, we will delve deeply into p-values and power.
A discussion on hypothesis testing, particularly Type I and Type II errors, would not be complete without talking about sample size. When making inferences about a population, a researcher collects a random sample of observations to determine whether there are statistical differences in each population. A random sample is one in which sample observations have an equal chance of being chosen. Sampling techniques have been developed to choose random samples, such as random value tables and statistical software. Choosing a large enough sample size avoids Type I and Type II errors.
We have concluded our discussion on hypothesis testing by looking at Type I and Type II errors. Type I errors are “false positives,” and Type II errors are “false negatives.” Both errors can lead to wrong conclusions about a hypothesis. Every data scientist should keep these concepts in mind when performing statistical inference.